141 research outputs found
Non-parametric Bayesian modeling of complex networks
Modeling structure in complex networks using Bayesian non-parametrics makes
it possible to specify flexible model structures and infer the adequate model
complexity from the observed data. This paper provides a gentle introduction to
non-parametric Bayesian modeling of complex networks: Using an infinite mixture
model as running example we go through the steps of deriving the model as an
infinite limit of a finite parametric model, inferring the model parameters by
Markov chain Monte Carlo, and checking the model's fit and predictive
performance. We explain how advanced non-parametric models for complex networks
can be derived and point out relevant literature
Bayesian Dropout
Dropout has recently emerged as a powerful and simple method for training
neural networks preventing co-adaptation by stochastically omitting neurons.
Dropout is currently not grounded in explicit modelling assumptions which so
far has precluded its adoption in Bayesian modelling. Using Bayesian entropic
reasoning we show that dropout can be interpreted as optimal inference under
constraints. We demonstrate this on an analytically tractable regression model
providing a Bayesian interpretation of its mechanism for regularizing and
preventing co-adaptation as well as its connection to other Bayesian
techniques. We also discuss two general approximate techniques for applying
Bayesian dropout for general models, one based on an analytical approximation
and the other on stochastic variational techniques. These techniques are then
applied to a Baysian logistic regression problem and are shown to improve
performance as the model become more misspecified. Our framework roots dropout
as a theoretically justified and practical tool for statistical modelling
allowing Bayesians to tap into the benefits of dropout training.Comment: 21 pages, 3 figures. Manuscript prepared 2014 and awaiting submissio
The Infinite Degree Corrected Stochastic Block Model
In Stochastic blockmodels, which are among the most prominent statistical
models for cluster analysis of complex networks, clusters are defined as groups
of nodes with statistically similar link probabilities within and between
groups. A recent extension by Karrer and Newman incorporates a node degree
correction to model degree heterogeneity within each group. Although this
demonstrably leads to better performance on several networks it is not obvious
whether modelling node degree is always appropriate or necessary. We formulate
the degree corrected stochastic blockmodel as a non-parametric Bayesian model,
incorporating a parameter to control the amount of degree correction which can
then be inferred from data. Additionally, our formulation yields principled
ways of inferring the number of groups as well as predicting missing links in
the network which can be used to quantify the model's predictive performance.
On synthetic data we demonstrate that including the degree correction yields
better performance both on recovering the true group structure and predicting
missing links when degree heterogeneity is present, whereas performance is on
par for data with no degree heterogeneity within clusters. On seven real
networks (with no ground truth group structure available) we show that
predictive performance is about equal whether or not degree correction is
included; however, for some networks significantly fewer clusters are
discovered when correcting for degree indicating that the data can be more
compactly explained by clusters of heterogenous degree nodes.Comment: Originally presented at the Complex Networks workshop NIPS 201
Neural Message Passing with Edge Updates for Predicting Properties of Molecules and Materials
Neural message passing on molecular graphs is one of the most promising
methods for predicting formation energy and other properties of molecules and
materials. In this work we extend the neural message passing model with an edge
update network which allows the information exchanged between atoms to depend
on the hidden state of the receiving atom. We benchmark the proposed model on
three publicly available datasets (QM9, The Materials Project and OQMD) and
show that the proposed model yields superior prediction of formation energies
and other properties on all three datasets in comparison with the best
published results. Furthermore we investigate different methods for
constructing the graph used to represent crystalline structures and we find
that using a graph based on K-nearest neighbors achieves better prediction
accuracy than using maximum distance cutoff or the Voronoi tessellation graph
Nonnegative Matrix Factorization with Gaussian Process Priors
We present a general method for including prior knowledge in a nonnegative matrix factorization (NMF), based on Gaussian process priors.
We assume that the nonnegative factors in the NMF are linked by a
strictly increasing function to an underlying Gaussian process specified
by its covariance function. This allows us to find NMF decompositions
that agree with our prior knowledge of the distribution of the factors, such
as sparseness, smoothness, and symmetries. The method is demonstrated
with an example from chemical shift brain imaging
- …